<html>
<head><meta charset="utf-8"><title>Memory profiling tooling · t-compiler/shrinkmem-sprint · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/index.html">t-compiler/shrinkmem-sprint</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html">Memory profiling tooling</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="226379434"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226379434" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rylev <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226379434">(Feb 15 2021 at 12:45)</a>:</h4>
<p><span class="user-mention" data-user-id="116083">@pnkfelix</span> Are you hoping to find a way to profile the rustc compilation process explicitly or to collect system wide profiling analytics which allow you to dive into the specifics of the rustc process? On Windows, I'm hoping to be able to gather analytics directly related to rustc, but I'll mostly be going through system wide analytics (through Windows Performance Recorder [WPR] and Windows Performance Analyzer [WPA]).</p>



<a name="226380269"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226380269" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226380269">(Feb 15 2021 at 12:55)</a>:</h4>
<p>there might be interesting correlations system wide too. like for instance, paging file behaviour is closely but sometimes not obviously related to the memory use.</p>



<a name="226392482"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226392482" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226392482">(Feb 15 2021 at 14:48)</a>:</h4>
<p>I'm planning to focus on DHAT for now, which IIUC will only focus on a single process (or process tree rooted at that process? Not sure.) System-wide stuff might be interesting, and I want to spend time better understanding my options there, but I don't think I can guarantee have something up and running on that front this week.</p>



<a name="226392686"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226392686" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226392686">(Feb 15 2021 at 14:50)</a>:</h4>
<p>(I am skimming over parts of Brendan Gregg's book on "Systems Performance", which has been on my bookshelf for probably 4 or 5 years now, to get a better feel of what tools are available here.)</p>



<a name="226833315"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226833315" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226833315">(Feb 18 2021 at 16:03)</a>:</h4>
<p>so one obvious issue with us using <em>only</em> DHAT is that its not really appropriate for gather data on arbitrary runs</p>



<a name="226833348"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226833348" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226833348">(Feb 18 2021 at 16:03)</a>:</h4>
<p>due to the overhead it imposes</p>



<a name="226833951"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226833951" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226833951">(Feb 18 2021 at 16:05)</a>:</h4>
<p>it will probably be more appropriate to settle on distinct tools: a minimal overhead tool that we can apply aggressively (i.e. to every <code>rustc</code> run during <code>x.py build</code>), and a more invasive one like DHAT for when we identify a specific rustc invocation that we want more information out of</p>



<a name="226877127"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226877127" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226877127">(Feb 18 2021 at 20:43)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116083">pnkfelix</span> <a href="#narrow/stream/276895-t-compiler.2Fshrinkmem-sprint/topic/Memory.20profiling.20tooling/near/226833315">said</a>:</p>
<blockquote>
<p>so one obvious issue with us using <em>only</em> DHAT is that its not really appropriate for gather data on arbitrary runs</p>
</blockquote>
<p>oh, actually, this might have been … false?</p>



<a name="226877212"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226877212" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226877212">(Feb 18 2021 at 20:43)</a>:</h4>
<p>(I misinterpreted data from a run that I was doing overnight that got interrupted by my putting the computer to sleep without checking if it was still running the build!)</p>



<a name="226877462"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226877462" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226877462">(Feb 18 2021 at 20:45)</a>:</h4>
<p>At least, I just did a bootstrap where it took 6m30s to bootstrap without dhat, and a subsequent clean+build with <code>RUSTC_WRAPPER=dhat</code> took 7m01s.</p>



<a name="226878893"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226878893" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Tyson Nottingham <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226878893">(Feb 18 2021 at 20:55)</a>:</h4>
<p>The rustc-perf collector readme says DHAT and Massif both incur a 5-20x slowdown, and that seems right from what I've seen. I've tried running DHAT on rustc_middle compilation before, and gave up when it it was taking too long. I can't remember how long that was though. I've run Massif successfully on rustc_middle, FWIW.</p>



<a name="226880518"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226880518" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226880518">(Feb 18 2021 at 21:05)</a>:</h4>
<p>I must be doing something wrong with DHAT</p>



<a name="226880746"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226880746" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226880746">(Feb 18 2021 at 21:06)</a>:</h4>
<p>In hindsight, the data it is spitting out seems incomplete. I wasn't using <code>--trace-children=true</code>, but I don't think I should have to do that just to see the memory profile of the single <code>rustc</code> process..</p>



<a name="226880946"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226880946" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226880946">(Feb 18 2021 at 21:08)</a>:</h4>
<p>are you running rustc via the rustup shim?</p>



<a name="226881049"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226881049" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226881049">(Feb 18 2021 at 21:08)</a>:</h4>
<p>oh ... oh dear</p>



<a name="226881077"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226881077" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226881077">(Feb 18 2021 at 21:09)</a>:</h4>
<p>that must be what's happening</p>



<a name="226881253"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226881253" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226881253">(Feb 18 2021 at 21:10)</a>:</h4>
<p>but ... I thought the whole point is that x.py won't do that?</p>



<a name="226881290"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226881290" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226881290">(Feb 18 2021 at 21:10)</a>:</h4>
<p>wait, how are you running it?</p>



<a name="226881293"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226881293" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226881293">(Feb 18 2021 at 21:10)</a>:</h4>
<p>/me turns on <code>--verbose</code> mode</p>



<a name="226881315"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226881315" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226881315">(Feb 18 2021 at 21:10)</a>:</h4>
<p>x.py won't go through the rustup shim</p>



<a name="226881327"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226881327" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226881327">(Feb 18 2021 at 21:10)</a>:</h4>
<p>but it has its own shim</p>



<a name="226881373"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226881373" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226881373">(Feb 18 2021 at 21:11)</a>:</h4>
<p>ah</p>



<a name="226881387"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226881387" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226881387">(Feb 18 2021 at 21:11)</a>:</h4>
<p>if you want dhat or similar, I'd strongly recommend directly running build/x86_.../stage1/bin/rustc</p>



<a name="226881429"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226881429" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226881429">(Feb 18 2021 at 21:11)</a>:</h4>
<p>yeah okay</p>



<a name="226881478"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226881478" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226881478">(Feb 18 2021 at 21:11)</a>:</h4>
<p>I want to start with developing some recipe for gathering data per-crate</p>



<a name="226881489"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226881489" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226881489">(Feb 18 2021 at 21:11)</a>:</h4>
<p>with low-overhead</p>



<a name="226881628"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226881628" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226881628">(Feb 18 2021 at 21:12)</a>:</h4>
<p>within x.py?</p>



<a name="226881673"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226881673" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226881673">(Feb 18 2021 at 21:13)</a>:</h4>
<p>yes</p>



<a name="226881687"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226881687" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226881687">(Feb 18 2021 at 21:13)</a>:</h4>
<p>or</p>



<a name="226881698"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226881698" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226881698">(Feb 18 2021 at 21:13)</a>:</h4>
<p>at least by setting RUSTC_WRAPPER</p>



<a name="226881769"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226881769" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226881769">(Feb 18 2021 at 21:13)</a>:</h4>
<p>i.e. try to keep the current build workflow as unchanged as I can</p>



<a name="226881922"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226881922" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226881922">(Feb 18 2021 at 21:14)</a>:</h4>
<p>hm</p>



<a name="226881945"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226881945" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226881945">(Feb 18 2021 at 21:14)</a>:</h4>
<p><code>/usr/bin/time</code> might be fine</p>



<a name="226881999"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226881999" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226881999">(Feb 18 2021 at 21:15)</a>:</h4>
<p>we already have a mode to print time (via <code>std::time::instant</code>) in rustbuild</p>



<a name="226882005"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226882005" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226882005">(Feb 18 2021 at 21:15)</a>:</h4>
<p>er</p>



<a name="226882039"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226882039" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226882039">(Feb 18 2021 at 21:15)</a>:</h4>
<p>does it print RSS too?</p>



<a name="226882049"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226882049" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226882049">(Feb 18 2021 at 21:15)</a>:</h4>
<p><code>/usr/bin/time -v</code> will show memory info</p>



<a name="226882050"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226882050" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226882050">(Feb 18 2021 at 21:15)</a>:</h4>
<p>not currently, but we could add it</p>



<a name="226882086"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226882086" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226882086">(Feb 18 2021 at 21:15)</a>:</h4>
<p>/usr/bin/time is platform specific, I guess anything in rustbuild would be too</p>



<a name="226882103"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226882103" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226882103">(Feb 18 2021 at 21:16)</a>:</h4>
<p>not sure what's better</p>



<a name="226882204"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226882204" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226882204">(Feb 18 2021 at 21:16)</a>:</h4>
<p>tell people to install the GNU time?</p>



<a name="226882206"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226882206" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226882206">(Feb 18 2021 at 21:16)</a>:</h4>
<p>probably easier to give instructions if it's integrated and we support whatever is needed for windows, macos, and linux to get rss data</p>



<a name="226882223"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226882223" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226882223">(Feb 18 2021 at 21:16)</a>:</h4>
<p>but yeah, I think you're right: Add it to rustbuild</p>



<a name="226882340"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226882340" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226882340">(Feb 18 2021 at 21:17)</a>:</h4>
<p>I would add as a separate flag from the current time one (but potentially also something we enable on perf.rlo etc); the time output is parsed in theory by some scripts, and certainly by perf.rlo, so I wouldn't want to change it ad-hoc too much</p>



<a name="226882431"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226882431" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226882431">(Feb 18 2021 at 21:18)</a>:</h4>
<p>right, I won't perturb current output format.</p>



<a name="226885507"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226885507" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226885507">(Feb 18 2021 at 21:41)</a>:</h4>
<p><span class="user-mention" data-user-id="116122">@simulacrum</span> is RUSTC_PRINT_STEP_TIMINGS the thing you're referencing above?</p>



<a name="226885517"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226885517" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226885517">(Feb 18 2021 at 21:41)</a>:</h4>
<p>yes</p>



<a name="226885532"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226885532" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226885532">(Feb 18 2021 at 21:41)</a>:</h4>
<p>I think there's a config.toml option for it</p>



<a name="226885555"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226885555" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226885555">(Feb 18 2021 at 21:41)</a>:</h4>
<p>we just pass state to the shim scripts via environment</p>



<a name="226885668"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226885668" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226885668">(Feb 18 2021 at 21:42)</a>:</h4>
<p>oh, you're right, there's a <code>print_step_timings</code> key mentioned in bootstrap/config.rs</p>



<a name="226888465"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226888465" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226888465">(Feb 18 2021 at 22:01)</a>:</h4>
<p>hmm, I had forgotten that <code>-Ztime-passes</code> already spits out rss info</p>



<a name="226888566"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226888566" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226888566">(Feb 18 2021 at 22:01)</a>:</h4>
<p>(at a finer-grain, but still)</p>



<a name="226889211"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226889211" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226889211">(Feb 18 2021 at 22:04)</a>:</h4>
<p>I think only on linux?</p>



<a name="226889307"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226889307" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226889307">(Feb 18 2021 at 22:04)</a>:</h4>
<p>that might explain why I've never seen it (since I've tended to use <code>-Ztime-passes</code> on my mac alone), I think</p>



<a name="226890338"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226890338" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226890338">(Feb 18 2021 at 22:09)</a>:</h4>
<p>rustc_data_structures::profiling::get_resident_set_size looks like its trying to do <em>something</em> on Windows</p>



<a name="226890678"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226890678" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226890678">(Feb 18 2021 at 22:10)</a>:</h4>
<p>ah, I see: that function <em>won't</em> work for "Mac OS X", because its leveraging the <code>/proc</code> file system</p>



<a name="226890853"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226890853" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226890853">(Feb 18 2021 at 22:11)</a>:</h4>
<p>but I could replace it with a <code>getrusage</code> call</p>



<a name="226890882"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226890882" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226890882">(Feb 18 2021 at 22:11)</a>:</h4>
<p>there must already be a crate for this</p>



<a name="226890915"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226890915" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226890915">(Feb 18 2021 at 22:11)</a>:</h4>
<p>rustc-perf handrolls it</p>



<a name="226890959"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226890959" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226890959">(Feb 18 2021 at 22:11)</a>:</h4>
<p>with getrusage</p>



<a name="226891094"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226891094" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226891094">(Feb 18 2021 at 22:12)</a>:</h4>
<p>oh, I'm having flashbacks to my PhD days</p>



<a name="226891120"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226891120" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226891120">(Feb 18 2021 at 22:12)</a>:</h4>
<p>"probably misreports RSS on macOS"</p>



<a name="226891148"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226891148" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226891148">(Feb 18 2021 at 22:12)</a>:</h4>
<p>(from description for <a href="https://crates.io/crates/getr">https://crates.io/crates/getr</a> )</p>



<a name="226891320"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226891320" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226891320">(Feb 18 2021 at 22:13)</a>:</h4>
<p>oh dear</p>



<a name="226891351"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226891351" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226891351">(Feb 18 2021 at 22:13)</a>:</h4>
<p>they didn't even try to make the struct OS speciifc</p>



<a name="226891489"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226891489" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226891489">(Feb 18 2021 at 22:14)</a>:</h4>
<p><a href="https://docs.rs/sysinfo/0.16.3/sysinfo/trait.ProcessExt.html#tymethod.memory">https://docs.rs/sysinfo/0.16.3/sysinfo/trait.ProcessExt.html#tymethod.memory</a> is <em>very</em> heavy as a dependency I imagine, but I guess is cross platform :)</p>



<a name="226891688"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226891688" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226891688">(Feb 18 2021 at 22:15)</a>:</h4>
<p>I have to go, but I'll look more at this later.</p>



<a name="226951413"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226951413" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> The 8472 <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226951413">(Feb 19 2021 at 11:34)</a>:</h4>
<p>To record large allocations one could also lower the size at which the malloc implementation does a direct mmap allocation and then perf record mmap syscalls.</p>



<a name="226954750"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226954750" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tm <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226954750">(Feb 19 2021 at 12:11)</a>:</h4>
<p>rustc compiles jemalloc with stats enabled, so that would be another potential source of information (there is also memory profiling support, but disabled by default).</p>



<a name="226982667"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226982667" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226982667">(Feb 19 2021 at 15:53)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116083">pnkfelix</span> <a href="#narrow/stream/276895-t-compiler.2Fshrinkmem-sprint/topic/Memory.20profiling.20tooling/near/226891351">said</a>:</p>
<blockquote>
<p>they didn't even try to make the struct OS speciifc</p>
</blockquote>
<p>Is this struct actually defined with a common type (or at least prefix) on Linux and Mac? That does not match my recollection from over a decade ago, but maybe I'm thinking of a different struct</p>



<a name="226983033"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/226983033" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#226983033">(Feb 19 2021 at 15:55)</a>:</h4>
<p>seems like its at least worth taking a shot here</p>



<a name="227027334"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/227027334" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#227027334">(Feb 19 2021 at 20:59)</a>:</h4>
<p>how it started: “worth taking a shot”. how its going: "[RUSTC-TIMING] core test:false 32.757  max rss (kb): 121542246"</p>



<a name="227027541"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/227027541" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#227027541">(Feb 19 2021 at 21:00)</a>:</h4>
<p>To be clear, maxrss report seems okay on Linux. (Above is from my Mac, <em>after</em> dividing by 1024 to correct for the bytes/kb discrepancy between Linux+Mac. So, something is still wrong.)</p>



<a name="227038242"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/227038242" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#227038242">(Feb 19 2021 at 22:15)</a>:</h4>
<p><span class="user-mention" data-user-id="116083">@pnkfelix</span> imo we should report in bytes and do that multiplication in the code</p>



<a name="227038257"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/227038257" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#227038257">(Feb 19 2021 at 22:15)</a>:</h4>
<p>it's been a long-standing annoyance that perf.rlo doesn't</p>



<a name="227049876"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/227049876" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#227049876">(Feb 20 2021 at 00:16)</a>:</h4>
<p>I don’t have a preference about whether the metric is reported in kb or in bytes. Just needs to be consistent (or, if its inconsistent, then the unit needs to be attached to the value as a suffix)</p>



<a name="228227324"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/228227324" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#228227324">(Mar 01 2021 at 13:13)</a>:</h4>
<p>(for people following along: my PR <a href="https://github.com/rust-lang/rust/issues/82532">#82532</a> "Add <code>build.print_step_rusage</code> to config.toml" landed an hour ago. I suspect it only works reliably on Linux, as noted above.)</p>



<a name="228712550"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/228712550" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#228712550">(Mar 04 2021 at 03:58)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116083">pnkfelix</span> <a href="#narrow/stream/276895-t-compiler.2Fshrinkmem-sprint/topic/Memory.20profiling.20tooling/near/226833315">said</a>:</p>
<blockquote>
<p>so one obvious issue with us using <em>only</em> DHAT is that its not really appropriate for gather data on arbitrary runs</p>
</blockquote>
<p>some concrete data here: Running dhat on the compile of <code>rustc_query_impl</code> took a little less than ... 6 hours on my machine.</p>



<a name="228712593"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/228712593" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#228712593">(Mar 04 2021 at 03:59)</a>:</h4>
<p>(it normally takes about 80 seconds.)</p>



<a name="228712712"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/228712712" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#228712712">(Mar 04 2021 at 04:00)</a>:</h4>
<p>so if my math is right, that's a 270x overhead. I wonder if something's wrong, or if the overhead is not O(1).</p>



<a name="228781415"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/228781415" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#228781415">(Mar 04 2021 at 14:30)</a>:</h4>
<p>Decided to try a similar experiment of running <code>massif</code> on the compile of <code>rustc_query_impl</code>. The good news is that this "finished" in a little over 2 hours. The bad news is I put "finished" in quotes because it didn't complete successfully: Valgrind signaled an Internal Error:</p>



<a name="228781510"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/228781510" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#228781510">(Mar 04 2021 at 14:31)</a>:</h4>
<div class="codehilite"><pre><span></span><code>--776900-- VALGRIND INTERNAL ERROR: Valgrind received a signal 11 (SIGSEGV) - exiting
--776900-- si_code=2;  Faulting address: 0x104E5DCFB8;  sp: 0x104bcced30

valgrind: the &#39;impossible&#39; happened:
   Killed by fatal signal
</code></pre></div>



<a name="228781879"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/228781879" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#228781879">(Mar 04 2021 at 14:33)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116083">pnkfelix</span> <a href="#narrow/stream/276895-t-compiler.2Fshrinkmem-sprint/topic/Memory.20profiling.20tooling/near/228712712">said</a>:</p>
<blockquote>
<p>so if my math is right, that's a 270x overhead. I wonder if something's wrong, or if the overhead is not O(1).</p>
</blockquote>
<p>DHAT has definitely taken outrageous amounts of time to run on rustdoc before</p>



<a name="228782076"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory%20profiling%20tooling/near/228782076" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/276895-t-compiler/shrinkmem-sprint/topic/Memory.20profiling.20tooling.html#228782076">(Mar 04 2021 at 14:34)</a>:</h4>
<p><a href="#narrow/stream/122651-general/topic/DHAT.20and.20rustdoc/near/219669766">https://rust-lang.zulipchat.com/#narrow/stream/122651-general/topic/DHAT.20and.20rustdoc/near/219669766</a></p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>